Input display device and input display method

ABSTRACT

An input display device includes a display unit that displays a screen for handwriting input on an input screen including a plurality of input fields, a stroke data processing unit that groups stroke data which is input to a capture screen by handwriting into stroke data representing characters to generate grouped stroke data as grouping stroke data, a character recognition unit that conducts character recognition on the grouping stroke data to convert the grouping stroke data into at least one recognized character, and a control processing unit that displays the at least one recognized character at the plurality of input fields of the input screen correlated with positions, at which handwriting input was performed, in the screen for handwriting input.

BACKGROUND 1. Field

The present disclosure relates to an input display device that displayscharacters which are inputted by handwriting by a user and an inputdisplay method for displaying the characters.

2. Description of the Related Art

In general, a technique for generating stroke data is known (forexample, Japanese Unexamined Patent Application Publication No.11-259598). In the technique described in Japanese Unexamined PatentApplication Publication No. 11-259598, stroke data is generated when theuser fills in a form displayed on a screen by handwriting and then thestroke data is displayed on the screen as handwritten characters.

A technique for converting stroke data into recognized characters isknown (for example, Japanese Patent No. 3855696). In the techniquedescribed in Japanese Patent No. 3855696, when the user fills in a formdisplayed on a screen by handwriting, character recognition is conductedand the stroke data is converted into characters to be displayed on thescreen.

However, in the technique described in Japanese Patent No. 3855696, therecognized characters are displayed on the screen each time when theuser performs handwriting on the screen. That is, in the techniquedescribed in Japanese Patent No. 3855696, the user has to performhandwriting for every input fields in the form to fill out.

For example, in a case where a document displayed on a screen is a formthat has input fields in which a user's name or the like is to bewritten, the user wants to perform handwriting for all input fields atonce.

SUMMARY

It is desirable to provide an input display device and an input displaymethod that the user may suitably perform handwriting input.

According to an aspect of the disclosure, there is provided an inputdisplay device which includes a display unit that displays a screen forhandwriting input on an input screen including a plurality of inputfields, a stroke data processing unit that groups stroke data whenhandwriting input is performed on the screen for handwriting input intostroke data representing a character string to generate grouped strokedata as grouping stroke data, a character recognition unit that conductscharacter recognition on the grouping stroke data and converts thegrouping stroke data into at least one recognized character, and acontrol processing unit that displays the at least one recognizedcharacter at the plurality of input fields of the input screencorrelated with positions, at which handwriting input was performed inthe screen for handwriting input.

According to another aspect of the disclosure, there is provided aninput display method which includes displaying a screen for handwritinginput on an input screen including a plurality of input fields, groupingstroke data when handwriting input is performed on the screen forhandwriting input into stroke data representing a character string andgenerating grouped stroke data as grouping stroke data, conductingcharacter recognition on the grouping stroke data and converting thegrouping stroke data into at least one recognized character, anddisplaying the at least one recognized character at the plurality ofinput fields of the input screen correlated with positions, at whichhandwriting input was performed in the screen for handwriting input.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a configuration of an input displaydevice according to a first embodiment;

FIG. 2 is a flowchart illustrating an operation of the input displaydevice according to the first embodiment;

FIG. 3 is a flowchart illustrating a handwriting input mode as anoperation of the input display device according to the first embodiment;

FIG. 4 is a diagram for explaining the operation of the input displaydevice according to the first embodiment;

FIG. 5 is a diagram for explaining the operation of the input displaydevice according to the first embodiment;

FIG. 6 is a diagram for explaining the operation of the input displaydevice according to the first embodiment;

FIG. 7 is a diagram for explaining the operation of the input displaydevice according to the first embodiment;

FIG. 8 is a diagram for explaining the operation of the input displaydevice according to the first embodiment;

FIG. 9 is a diagram for explaining the operation of the input displaydevice according to the first embodiment;

FIG. 10 is a diagram for explaining the operation of the input displaydevice according to the first embodiment;

FIG. 11 is a diagram for explaining the operation of the input displaydevice according to the first embodiment;

FIG. 12 is a diagram for explaining the operation of the input displaydevice according to the first embodiment;

FIG. 13 is a diagram for explaining an operation of an input displaydevice according to a second embodiment;

FIG. 14 is a diagram for explaining the operation of the input displaydevice according to the second embodiment;

FIG. 15 is a diagram for explaining the operation of the input displaydevice according to the second embodiment;

FIG. 16 is a diagram for explaining the operation of the input displaydevice according to the second embodiment;

FIG. 17 is a diagram for explaining the operation of the input displaydevice according to the second embodiment;

FIG. 18 is a flowchart illustrating a handwriting input mode as anoperation of an input display device according to a third embodiment;

FIG. 19 is a flowchart illustrating the handwriting input mode as theoperation of the input display device according to the third embodiment;

FIG. 20 is a diagram for explaining the operation of the input displaydevice according to the third embodiment;

FIG. 21 is a diagram for explaining the operation of the input displaydevice according to the third embodiment;

FIG. 22 is a diagram for explaining the operation of the input displaydevice according to the third embodiment;

FIG. 23 is a diagram for explaining the operation of the input displaydevice according to the third embodiment;

FIG. 24 is a diagram for explaining the operation of the input displaydevice according to the third embodiment;

FIG. 25 is a diagram for explaining an operation of an input displaydevice according to a fourth embodiment;

FIG. 26 is a diagram for explaining the operation of the input displaydevice according to the fourth embodiment;

FIG. 27 is a diagram for explaining the operation of the input displaydevice according to the fourth embodiment;

FIG. 28 is a diagram for explaining an operation of an input displaydevice according to a seventh embodiment;

FIG. 29 is a diagram for explaining the operation of the input displaydevice according to the seventh embodiment;

FIG. 30 is a diagram for explaining an operation of an input displaydevice according to an eighth embodiment;

FIG. 31 is a diagram for explaining the operation of the input displaydevice according to the eighth embodiment;

FIG. 32 is a diagram for explaining the operation of the input displaydevice according to the eighth embodiment;

FIG. 33 is a diagram for explaining an operation of an input displaydevice according to a ninth embodiment;

FIG. 34 is a diagram for explaining the operation of the input displaydevice according to the ninth embodiment; and

FIG. 35 is a diagram for explaining the operation of the input displaydevice according to the ninth embodiment.

DESCRIPTION OF THE EMBODIMENTS

In the following, embodiments will be described with reference to theaccompanying drawings.

First Embodiment

FIG. 1 is a diagram illustrating a configuration of an input displaydevice 100 according to a first embodiment. The input display device 100includes a control unit 110, a storage unit 120, a display unit 130, ascreen processing unit 140, an input unit 150, a stroke data processingunit 160, and a character recognition unit 170.

The control unit 110 is a functional unit for controlling the entireinput display device 100. The control unit 110 reads and executesvarious computer programs (in the following, referred to as a program)stored in the storage unit 120 to thereby realize various functions andis configured with, for example, a central process unit (CPU).

The storage unit 120 is a functional unit that stores various programsor various pieces of data demanded for an operation of the input displaydevice 100. The storage unit 120 is configured with, for example, asemiconductor memory or a hard disk drive (HDD). The storage unit 120stores a coordinate table 500 which will be described later.

The display unit 130 is a touch panel display and is configured with,for example, a liquid crystal display (LCD) and a touch panel. Thescreen processing unit 140 is a control processing unit that performsprocessing for controlling the display of various screens on the displayunit 130.

The input unit 150 includes a keyboard 152, a pointing device 154, and atouch pen 156. For example, the user may perform handwriting input to ascreen using the pointing device 154 or the touch pen 156. The strokedata processing unit 160 generates characters which are inputted byhandwriting as stroke data which will be described later.

The character recognition unit 170 conducts optical characterrecognition (generally abbreviated as OCR) to at least one character(stroke data) which is inputted by handwriting and converts the at leastone character into the at least one recognized character which is acharacter capable of being edited by a computer.

The operation of the input display device 100 according to the firstembodiment will be described using FIGS. 2 to 11. FIG. 2 is a flowchartillustrating an operation of the input display device 100 according tothe first embodiment. FIG. 3 is a flowchart illustrating a handwritinginput mode as an operation of the input display device 100 according tothe first embodiment. FIGS. 4 to 11 are diagrams for explaining theoperation of the input display device 100 according to the firstembodiment.

First, a normal input mode will be described. The screen processing unit140 displays document data on the display unit 130 as an input screen200 (see FIG. 4) (Step S2 of FIG. 2).

As illustrated in FIG. 4, the input screen 200 (document data) includesa plurality of input fields 210 and 212 as input positions. The inputfields 210 and 212 are positions (fields) at which a last name and afirst name, which are a name of a user, are input by Chinese charactersand Hiragana characters, respectively.

The user selects the normal input mode using the input unit 150(keyboard 152) (No in Step S4 of FIG. 2). In this case, the normal inputmode is executed (Step S6 of FIG. 2).

In the normal input mode, the user inputs characters (character string)of “HATSUMEI” and “TARO” using the input unit 150 (keyboard 152) to theplurality of input fields 210 and 212 of the input screen 200 displayedon the display unit 130, respectively. As illustrated in FIG. 11, thescreen processing unit 140 displays the input screen 200, in whichcharacters of “HATSUMEI” and “TARO” are displayed at the respectiveinput fields 210 and 212, on the display unit 130.

Next, a handwriting input mode will be described. The screen processingunit 140 displays document data on the display unit 130 as the inputscreen 200 (see FIG. 4) (Step S2 of FIG. 2). Document data is datarepresenting a document by a hypertext markup language (HTML), aportable document format (PDF), or the like.

As illustrated in FIG. 5, in a coordinate table 500, data representingthe X coordinate and Y coordinate for the input screen 200 and the inputfields 210 and 212 are stored. Specifically, the input screen 200 iscorrelated with the minimum value “X0” of the X coordinate, the maximumvalue “Xmax” of the X coordinate, the minimum value “Y0” of the Ycoordinate, and the maximum value “Ymax” of the Y coordinate. The inputfield 210 is correlated with the minimum value “X2101” of the Xcoordinate, the maximum value “X2102” of the X coordinate, the minimumvalue “Y2101” of the Y coordinate, and the maximum value “Y2102” of theY coordinate. The input field 212 is correlated with the minimum value“X2121” of the X coordinate, the maximum value “X2122” of the Xcoordinate, the minimum value “Y2121” of the Y coordinate, and themaximum value “Y2122” of the Y coordinate.

The user selects the handwriting input mode using the input unit 150(pointing device 154 or touch pen 156) (Yes in Step S4 of FIG. 2). Inthis case, the handwriting input mode is executed (Step S8 of FIG. 2).

In the handwriting input mode, as illustrated in FIG. 6, the screenprocessing unit 140 captures the input screen 200 displayed on thedisplay unit 130 as a capture screen 300. The screen processing unit 140saves the capture screen 300 in the storage unit 120 and also displaysthe capture screen 300 on the input screen 200 (Step S100 of FIG. 3).

As illustrated in FIG. 7, the user inputs characters “HATSUMEI” and“TARO” in the plurality of input fields 210 and 212 of the capturescreen 300 by handwriting using the input unit 150 (pointing device 154or touch pen 156) (Step S102 of FIG. 3).

In this case, the user may input characters “HATSUMEI” and “TARO” byhandwriting by extending beyond the plurality of input fields 210 and212 of the capture screen 300. The stroke data processing unit 160generates a plurality of coordinate columns configuring inputcharacters, which are inputted to the capture screen 300 by handwriting,as stroke data, respectively, and saves the stroke data in the storageunit 120.

The user performs a recognition instruction using the input unit 150(pointing device 154 or touch pen 156) (Step S104 of FIG. 3).

In this case, as illustrated in FIG. 7, the stroke data processing unit160 groups stroke data saved in the storage unit 120 into stroke datarepresenting a character string correlated with the input fields 210 and212 and generates grouped stroke data as pieces of grouping stroke data310 and 312 (Step S106 of FIG. 3).

Step S106 will be described using FIG. 8. Each of pieces of groupingstroke data 310 and 312 represents pieces of stroke data spanning fromfirst stroke data SD1 to last stroke data SDn (n is an integer of 1 ormore) when the user input by handwriting. The pieces of stroke dataspanning from first stroke data SD1 to last stroke data SDn constitutesa single piece of grouping stroke data when a predetermined time Tpelapsed after the user finished handwriting input (for example, in acase of the touch pen 156, from the time when the pen is lifted up).

Here, in a technique for grouping the pieces of stroke data spanningfrom first stroke data SD1 to last stroke data SDn, position informationrepresenting a position when input was performed by handwriting may beused, instead of time information such as a predetermined time Tp. Forexample, clustering by a Ward method is known, as the technique forgrouping pieces of data using position information. In the groupingtechnique, time information and position information may be combinedwith each other to be used. In FIG. 8, although the character stringreferred to as “HATSUMEI” is grouped, grouping is performed on, forexample, “HATSU”, “MEI”, and each character and then, the characterstring referred to as “HATSUMEI” may be grouped.

The character recognition unit 170 conducts character recognition onpieces of grouping stroke data 310 and 312 and converts each of thepieces of grouping stroke data 310 and 312 into each of recognizedcharacters (recognized character string) 410 and 412 as characterscapable of being edited by a computer (Step S108 of FIG. 3). Therecognized characters of the present specification may be a singlecharacter or a plurality of characters and includes a character andcharacter string.

In this case, as illustrated in FIG. 9, the screen processing unit 140displays the recognized characters 410 and 412 on the position at whichinput was performed to the capture screen 300 by handwriting (Step S110of FIG. 3). The recognized characters 410 and 412 represent characters“HATSUMEI” and “TARO”, respectively.

Step S110 will be described using FIG. 10. The screen processing unit140 detects the coordinate of each of the pieces of grouping stroke data310 and 312. In a case where the grouping stroke data 310 is used as anexample, the screen processing unit 140 detects the minimum value“X3101” of the X coordinate, the maximum value “X3102” of the Xcoordinate, the minimum value “Y3101” of the Y coordinate, and themaximum value “Y3102” of the Y coordinate of the capture screen 300 forgrouping stroke data 310 and recognizes (X3101, Y3101) (X3102, Y3101)(X3101, Y3102) (X3102, Y3102) as a coordinate region XY 3100 of thecoordinates. The screen processing unit 140 changes (expands) acharacter size of the characters “HATSUMEI” which are the recognizedcharacters 410 from a set character size, which is preset, to acharacter size based on the coordinate region 3100. For example, thescreen processing unit 140 determines a magnification scale factor forthe set character size. The screen processing unit 140 displays thecharacters “HATSUMEI” which are the recognized characters 410 on thecoordinate region XY 3100 of the capture screen 300 in a character sizebased on the coordinate region 3100.

For example, in a case where it is determined that there is no error inat least one recognized character of the recognized characters 410 and412 displayed on the capture screen 300 (No in Step S112 of FIG. 3), theuser executes Step S116 which will be described later.

On the other hand, in a case where it is determined that there is anerror in at least one recognized character of the recognized characters410 and 412 displayed on the capture screen 300 (Yes in Step S112 ofFIG. 3), the user performs correction on the at least one recognizedcharacter (Step S114 of FIG. 3). Specifically, Steps S102, S104, S106,S108, and S110 are executed for the at least one recognized character.Otherwise, in Step S114, a soft keyboard is displayed on the capturescreen 300 and correction may be performed on the at least onerecognized character.

The user performs an input instruction using the input unit 150(pointing device 154 or touch pen 156) (Step S116 of FIG. 3).

In this case, as illustrated in FIG. 11, the screen processing unit 140displays the input screen 200 in which the recognized characters 410 and412 are displayed at the input fields 210 and 212, respectively, insteadof the capture screen 300 (Step S118 of FIG. 3). Characters “HATSUMEI”and “TARO” are displayed in the input fields 210 and 212 of the inputscreen 200, respectively.

Step S118 will be described using FIG. 12. The screen processing unit140 determines whether the coordinate region detected for the pieces ofgrouping stroke data 310 and 312 and the coordinate region of the inputfields 210 and 212 of the input screen 200 overlap each other,respectively, or not. In a case where grouping stroke data 310 is usedas an example, (X3101, Y3101) (X3102, Y3101) (X3101, Y3102) (X3102,Y3102) for grouping stroke data 310 is already recognized, as thecoordinate region XY 3100 of the capture screen 300, by the screenprocessing unit 140. The screen processing unit 140 references thecoordinate table 500 to thereby recognize (X2101, Y2101) (X2102, Y2101)(X2101, Y2102) (X2102, Y2102) as the coordinate region XY 2100 of theinput field 210 of the input screen 200.

As illustrated in FIG. 12, if X2101≤X3101≤X2102 and Y2101≤Y3101≤Y2102,or X2101≤X3102≤X2102 and Y2101≤Y3102≤Y2102, the coordinate region XY3100 of the capture screen 300 and the coordinate region XY 2100 of theinput field 210 of the input screen 200 overlap each other. In thiscase, the screen processing unit 140 changes (reduces) the charactersize of the characters “HATSUMEI” which are the recognized characters410 from a character size based on the coordinate region 3100 to acharacter size based on the coordinate region XY 2100. The screenprocessing unit 140 determines, for example, a reduction scale factorfor the character size based on the coordinate region 3100. The screenprocessing unit 140 displays the characters “HATSUMEI” which are therecognized characters 410 on the input field 210 of the input screen 200in the character size based on the coordinate region XY 2100.

Here, in a case where the coordinate region XY 3100 of the capturescreen 300 of a recognition result overlaps the coordinate region of theinput fields 210 and 212 of the input screen 200, the input field 210,which is nearest to the center of the coordinate region XY 3100 of therecognition result, of the input fields 210 and 212 of the input screen200 may be selected. In general, in a case where a horizontal writingcharacter is written by handwriting, writing is started from the leftupper corner. In a case where, the coordinate region XY 3100 of thecapture screen 300 of the recognition result overlaps the coordinateregion of the input fields 210 and 212 of the input screen 200, theinput field 210 at the far left and uppermost side of the input fields210 and 212 of the input screen 200 may be selected.

As described above, the input display device 100 according to the firstembodiment includes the display unit 130 that displays the input screen200 including the input fields 210 and 212, the screen processing unit140 that captures the input screen 200 displayed on the display unit 130as the screen for handwriting input (capture screen 300) and displaysthe capture screen 300 on the input screen 200, the stroke dataprocessing unit 160 that groups stroke data when input was performed onthe capture screen 300 by handwriting into stroke data which representsthe character string, which is in correlation with the input fields 210and 212, and generates the stroke data as pieces of grouping stroke data310 and 312, and the character recognition unit 170 that conductscharacter recognition on the pieces of grouping stroke data 310 and 312and converts the pieces of grouping stroke data 310 and 312 into therecognized characters 410 and 412. In this case, the characterrecognition unit 170 converts the pieces of grouping stroke data 310 and312 into the recognized characters 410 and 412 based on the input fields210 and 212 of the input screen 200 and the screen processing unit 140displays the recognized characters 410 and 412 at the input fields 210and 212 of the input screen 200. For that reason, in the input displaydevice 100 according to the first embodiment, the user may collectivelyperform handwriting input without causing the user to input thecharacter string by handwriting every time as in the technique describedin Japanese Patent No. 3855696.

Here, in the input display device 100 according to the first embodiment,in a case where pieces of grouping stroke data 310 and 312 are displacedwith respect to the input fields 210 and 212 of the input screen 200,the screen processing unit 140 displays the recognized characters 410and 412, which are obtained by conducting character recognition on thepieces of grouping stroke data 310 and 312, at the input fields 210 and212 based on the coordinate regions of the pieces of grouping strokedata 310 and 312 and the coordinate regions of the input fields 210 and212. For example, a portion of the coordinate regions of the pieces ofgrouping stroke data 310 and 312 overlaps the coordinate regions of theinput fields 210 and 212, but a portion of the coordinate regions of thepieces of grouping stroke data 310 and 312 is extended beyond thecoordinate regions of the input fields 210 and 212. Also, in this case,the recognized characters 410 and 412 obtained by conducting characterrecognition on the pieces of grouping stroke data 310 and 312 may bedisplayed at the input fields 210 and 212 based on these coordinateregions.

In the present embodiment, although the at least one recognizedcharacter is displayed on the capture screen 300 once, the at least onerecognized character may also be output to an input field (for example,input form) of the input screen 200 as a simple text character(character string). Specifically, in a case where the input field 210 isan input field such as a text box in FIG. 11, a text character string of“HATSUMEI” may be output (handed over) to the input field.

That is, a text input field which is close to a position, at which thetext character is input as the input character, is recognized. The atleast one recognized character may also be output to the name (forexample, if an HTML file is present, a text box specified by “<inputtype=“text” name=“abc”>”) of the text input field as a text.

Second Embodiment

In the second embodiment, for example, an input field for receiving afull width Katakana character or a half width numeral is included in theinput screen 200, as restriction information restricting a type of acharacter.

The operations of the input display device 100 according to the secondembodiment will be described using FIGS. 2, 3, and 13 to 17. FIGS. 13 to17 are diagrams for explaining the operations of the input displaydevice 100 according to the second embodiment.

First, a normal input mode will be described. The screen processing unit140 displays document data on the display unit 130 as the input screen200 (see FIG. 13) (Step S2 of FIG. 2).

As illustrated in FIG. 13, the input screen 200 (document data) alsoincludes a plurality of input fields 220, 222, 230, 232, and 234. Theinput fields 220 and 222 are Katakana input fields for receiving thelast name and first name which are a name of the user using Katakanacharacter, respectively. The input fields 230, 232, and 234 are numericinput fields for receiving a year (A.D.), month, and day which are dateof birth of the user using a numeral, respectively.

As illustrated in FIG. 14, the coordinate table 500 also stores datarepresenting the X coordinate and the Y coordinate for the input fields220 and 222, 230, 232, and 234. Specifically, the input field 220 iscorrelated with the minimum value “X2201” of the X coordinate, themaximum value “X2202” of the X coordinate, the minimum value “Y2201” ofthe Y coordinate, and the maximum value “Y2202” of the Y coordinate. Theinput field 222 is correlated with the minimum value “X2221” of the Xcoordinate, the maximum value “X2222” of the X coordinate, the minimumvalue “Y2221” of the Y coordinate, and the maximum value “Y2222” of theY coordinate. The input field 230 is correlated with the minimum value“X2301” of the X coordinate, the maximum value “X2302” of the Xcoordinate, the minimum value “Y2301” of the Y coordinate, and themaximum value “Y2302” of the Y coordinate. The input field 232 iscorrelated with the minimum value “X2321” of the X coordinate, themaximum value “X2322” of the X coordinate, the minimum value “Y2321” ofthe Y coordinate, and the maximum value “Y2322” of the Y coordinate. Theinput field 234 is correlated with the minimum value “X2341” of the Xcoordinate, the maximum value “X2342” of the X coordinate, the minimumvalue “Y2341” of the Y coordinate, and the maximum value “Y2342” of theY coordinate.

As illustrated in FIG. 14, the coordinate table 500 stores restrictioninformation restricting a type of a character for the input fields 220and 222, 230, 232, and 234. Specifically, the input fields 220 and 222are correlated with Katakana restriction information 520 for causing theuser to input a full width Katakana character as the type of thecharacter. The input fields 230, 232, and 234 are correlated withnumeric restriction information 530 for causing the user to input a halfwidth numeral as the type of the character. In a case where thecoordinate table 500 is referenced and Katakana restriction information520 is correlated with the input fields 220 and 222, the characterrecognition unit 170 recognizes that the input fields 220 and 222 forcausing the user to input a full width Katakana character are present inthe input screen 200. In a case where the coordinate table 500 isreferenced and numeric restriction information 530 is correlated withthe input fields 230, 232, and 234, the character recognition unit 170recognizes that the input fields 230, 232, and 234 for causing the userto input a half width numeral are present in the input screen 200.

The user selects the normal input mode by using the input unit 150(keyboard 152) (No in Step S4 of FIG. 2). In this case, the normal inputmode is executed (Step S6 of FIG. 2).

In the normal input mode, the user inputs characters “HATSUMEI”, “TARO”,“hatsumei”, “taro”, “2016”, “1”, and “1” in the input fields 210 212,220, 222, 230, 232, and 234 of the input screen 200 displayed on thedisplay unit 130, respectively, by using the input unit 150 (keyboard152). As illustrated in FIG. 11, the screen processing unit 140 displaysthe input screen 200, in which characters “HATSUMEI” and “TARO”,characters “hatsumei” and “taro” by a full width Katakana character, andcharacters “2016”, “1”, and “1” by a half width numeral are displayed atthe input fields 210 212, 220, 222, 230, 232, and 234, respectively, onthe display unit 130.

Next, a handwriting input mode will be described. The screen processingunit 140 displays document data on the display unit 130 as the inputscreen 200 (see FIG. 13) (Step S2 of FIG. 2). Document data is datarepresenting a document by a hypertext markup language (HTML), aportable document format (PDF), or the like.

The user selects the handwriting input mode using the input unit 150(pointing device 154 or touch pen 156) (Yes in Step S4 of FIG. 2). Inthis case, the handwriting input mode is executed (Step S8 of FIG. 2).

In the handwriting input mode, the screen processing unit 140 capturesthe input screen 200 displayed on the display unit 130 as a capturescreen 300 (see FIG. 15), and saves the capture screen 300 in thestorage unit 120 and also displays the capture screen 300 on the inputscreen 200 (Step S100 of FIG. 3).

As illustrated in FIG. 15, the user inputs characters “HATSUMEI”,“TARO”, “hatsumei”, “taro”, “2016”, “1”, and “1” to the input fields210, 212, 220, 222, 230, 232, and 234 of the capture screen 300 byhandwriting using the input unit 150 (pointing device 154 or touch pen156) (Step S102 of FIG. 3).

In this case, the user inputs characters “HATSUMEI”, “TARO”, “hatsumei”,“taro”, “2016”, “1”, and “1” by handwriting by extending beyond theinput fields 210, 212, 220, 222, 230, 232, and 234 of the capture screen300. The stroke data processing unit 160 generates coordinate columnsconfiguring characters, which are inputted to the capture screen 300 byhandwriting, as stroke data, respectively, and saves the stroke data inthe storage unit 120.

The user performs a recognition instruction using the input unit 150(pointing device 154 or touch pen 156) (Step S104 of FIG. 3).

In this case, as illustrated in FIG. 15, the stroke data processing unit160 groups stroke data saved in the storage unit 120 into stroke datarepresenting a character string correlated with the input fields 210 and212, 220, 222, 230, 232, and 234 and generates grouped stroke data aspieces of grouping stroke data 310, 312, 320, 322, 330, 332, and 334(Step S106 of FIG. 3).

The character recognition unit 170 conducts character recognition onpieces of grouping stroke data 310, 312, 320, 322, 330, 332, and 334 andconverts each of pieces of grouping stroke data 310, 312, 320, 322, 330,332, and 334 into each of recognized characters 410, 412, 420, 422, 430,432, and 434 as characters capable of being edited by a computer (StepS108 of FIG. 3).

In this case, Katakana restriction information 520 is in correlationwith the input fields 220 and 222 of the coordinate table 500 and thus,the character recognition unit 170 recognizes that the input fields 220and 222 for causing the user to input a full width Katakana characterare present in the input screen 200 to convert the recognized characters420 and 422 into recognized characters 420 and 422 that represent a fullwidth Katakana character. Numeric restriction information 530 is incorrelation with the input fields 230, 232, and 234 of the coordinatetable 500 and thus, the character recognition unit 170 recognizes thatthe input fields 230, 232, and 234 for causing the user to input a halfwidth numeral are present in the input screen 200 to convert therecognized characters 430, 432, and 434 into the recognized characters430, 432, and 434 that represent a half width numeral.

In this case, as illustrated in FIG. 16, the screen processing unit 140displays the recognized characters 410, 412, 420, 422, 430, 432, and 434on the position at which input was performed to the capture screen 300by handwriting (Step S110 of FIG. 3). The recognized characters 410,412, 420, 422, 430, 432, and 434 represent characters “HATSUMEI” and“TARO”, characters “hatsumei” and “taro” by a full width Katakanacharacter, and characters “2016”, “1”, and “1” by a half width numeral,respectively.

For example, in a case where it is determined that there is no error inat least one recognized character of the recognized characters 410, 412,420, 422, 430, 432, and 434 displayed on the capture screen 300 (No inStep S112 of FIG. 3), the user executes Step S116 which will bedescribed later.

On the other hand, in a case where it is determined that there is anerror in at least one recognized character of the recognized characters410, 412, 420, 422, 430, 432, and 434 displayed on the capture screen300 (Yes in Step S112 of FIG. 3), the user performs correction on the atleast one recognized character (Step S114 of FIG. 3). Specifically,Steps S102, S104, S106, S108, and S110 are executed for the at least onerecognized character. Otherwise, in Step S114, correction may beperformed on the at least one recognized character by displaying a softkeyboard on the capture screen 300.

The user performs an input instruction using the input unit 150(pointing device 154 or touch pen 156) (Step S116 of FIG. 3).

In this case, as illustrated in FIG. 17, the screen processing unit 140displays the input screen 200 in which the recognized characters 410,412, 420, 422, 430, 432, and 434 are displayed at the input fields 210212, 220, 222, 230, 232, and 234, respectively, instead of the capturescreen 300 (Step S118 of FIG. 3). Characters “HATSUMEI” and “TARO”,characters “hatsumei” and “taro” by a full width Katakana character, andcharacters “2016”, “1”, and “1” by a half width numeral are displayed onthe input fields 210 212, 220, 222, 230, 232, and 234 of the inputscreen 200, respectively.

The input fields 230, 232 and 234 of the input screen 200 are fields forreceiving a numeral and thus, may be a pull-down system that a user mayselect an arbitrary numeral.

As described above, the input display device 100 according to the secondembodiment includes the display unit 130 that displays the input screen200 including the input fields 210, 212, 220, 222, 230, 232, and 234,the screen processing unit 140 that captures the input screen 200displayed on the display unit 130 as the screen for handwriting input(capture screen 300) and displays the capture screen 300 on the inputscreen 200, the stroke data processing unit 160 that groups stroke datawhen input was performed on the capture screen 300 by handwriting intostroke data which represents the character string, which is incorrelation with the input fields 210, 212, 220, 222, 230, 232, and 234,and generates the stroke data as pieces of grouping stroke data 310,312, 320, 322, 330, 332, and 334, and the character recognition unit 170that conducts character recognition on the pieces of grouping strokedata 310, 312, 320, 322, 330, 332, and 334 and converts the pieces ofgrouping stroke data 310, 312, 320, 322, 330, 332, and 334 into therecognized characters 410, 412, 420, 422, 430, 432, and 434. In thiscase, the character recognition unit 170 converts the pieces of groupingstroke data 310, 312, 320, 322, 330, 332, and 334 into the recognizedcharacters 410, 412, 420, 422, 430, 432, and 434 based on the inputfields 210, 212, 220, 222, 230, 232, and 234 of the input screen 200 andthe screen processing unit 140 displays the recognized characters 410,412, 420, 422, 430, 432, and 434 at the input fields 210, 212, 220, 222,230, 232, and 234 of the input screen 200, respectively. For thatreason, in the input display device 100 according to the secondembodiment, the user may collectively perform handwriting input withoutcausing the user to input the character string by handwriting every timeas in the technique described in Japanese Patent No. 3855696.

In the input display device 100 according to the second embodiment,pieces of restriction information 520 and 530 for restricting the typeof the character are correlated with at least one of the input fields220, 222, 230, 232, and 234 among the input fields 210, 212, 220, 222,230, 232, and 234. The character recognition unit 170 converts pieces ofgrouping stroke data 310, 312, 320, 322, 330, 332, and 334 into therecognized characters 410, 412, 420, 422, 430, 432, and 434 based on theinput fields 210, 212, 220, 222, 230, 232, and 234 and pieces ofrestriction information 520 and 530. For that reason, in the inputdisplay device 100 according to the second embodiment, a full widthKatakana character may be displayed at the input fields 220 and 222 or ahalf width numeral may be displayed at the input fields 230, 232, and234, as restriction information restricting the type of the character.

Here, in the input display device 100 according to the secondembodiment, even in a case where pieces of grouping stroke data 310,312, 320, 322, 330, 332, and 334 are displaced with respect to the inputfields 210, 212, 220, 222, 230, 232, and 234 of the input screen 200,the screen processing unit 140 displays the recognized characters 410and 412, 420, 422, 430, 432, and 434 obtained by conducting characterrecognition on the pieces of grouping stroke data 310, 312, 320, 322,330, 332, and 334 at the input fields 210, 212, 220, 222, 230, 232, and234 based on the coordinate regions of the pieces of grouping strokedata 310, 312, 320, 322, 330, 332, and 334, the coordinate regions ofthe input fields 210, 212, 220, 222, 230, 232, and 234, and pieces ofrestriction information 520 and 530. For example, a portion of thecoordinate regions of pieces of grouping stroke data 320 and 322 amongthe pieces of grouping stroke data 310, 312, 320, 322, 330, 332, and 334overlaps the coordinate regions of the input fields 220 and 222 amongthe input fields 210, 212, 220, 222, 230, 232, and 234, but a portion ofthe coordinate regions of pieces of grouping stroke data 320 and 322among the pieces of grouping stroke data 310, 312, 320, 322, 330, 332,and 334 is extended beyond the coordinate regions of the input fields220 and 222 among the input fields 210, 212, 220, 222, 230, 232, and234. Also, in this case, the recognized characters 420 and 422 obtainedby conducting character recognition on the pieces of grouping strokedata 320 and 322 may be displayed at the input fields 210 and 212 basedon the coordinate region and restriction information (Katakanarestriction information 520). For example, a portion of the coordinateregions of pieces of grouping stroke data 330, 332, and 334 among thepieces of grouping stroke data 310, 312, 320, 322, 330, 332, and 334overlaps the coordinate regions of the input fields 230, 232, and 234among the input fields 210, 212, 220, 222, 230, 232, and 234, but aportion of the coordinate regions of pieces of grouping stroke data 330,332, and 334 among the pieces of grouping stroke data 310, 312, 320,322, 330, 332, and 334 is extended beyond the coordinate regions of theinput fields 230, 232, and 234 among the input fields 210, 212, 220,222, 230, 232, and 234. Also, in this case, the recognized characters430, 432, and 434 obtained by conducting character recognition on thepieces of grouping stroke data 330, 332, and 334 may be displayed at theinput fields 230, 232, and 234 based on the coordinate region andrestriction information (numeric restriction information 530).

Third Embodiment

In the third embodiment, for example, an input field for allowing a userto select (designate) is included in the input screen 200, as attributeinformation.

The operations of the input display device 100 according to the thirdembodiment will be described using FIGS. 2 and 18 to 24. FIGS. 18 and 19are flowcharts illustrating a handwriting input mode as an operation ofthe input display device 100 according to the third embodiment. FIGS. 20to 24 are diagrams for explaining the operation of the input displaydevice 100 according to the third embodiment.

First, the normal input mode will be described. The screen processingunit 140 displays document data on the display unit 130 as the inputscreen 200 (see FIG. 20) (Step S2 of FIG. 2).

As illustrated in FIG. 20, the input screen 200 (document data) alsoincludes input fields 240 and 242. Each of the input fields 240 and 242is a check input field (so-called checkbox) for allowing a user toselect (designate) a male or a female as gender by attaching a checkmark (for example, “v”).

As illustrated in FIG. 21, the coordinate table 500 also stores datarepresenting the X coordinate and the Y coordinate for the input fields240 and 242. Specifically, the input field 240 is correlated with theminimum value “X2401” of the X coordinate, the maximum value “X2402” ofthe X coordinate, the minimum value “Y2401” of the Y coordinate, and themaximum value “Y2402” of the Y coordinate. The input field 242 iscorrelated with the minimum value “X2421” of the X coordinate, themaximum value “X2422” of the X coordinate, the minimum value “Y2421” ofthe Y coordinate, and the maximum value “Y2422” of the Y coordinate.

As illustrated in FIG. 21, the coordinate table 500 stores attributeinformation for the input fields 240 and 242. Specifically, the inputfields 240 and 242 are correlated with check mark attribute information540 for causing the user to designate (select) the input field 240 orthe input field 242. In a case where the coordinate table 500 isreferenced and check mark attribute information 540 is correlated withthe input fields 240 and 242, the character recognition unit 170, thecharacter recognition unit 170 recognizes that the input fields 240 and242 for causing the user to designate (select) the check mark arepresent in the input screen 200. That is, data which is input by aninput method using the check mark corresponds to attribute information.

The user selects the normal input mode by using the input unit 150(keyboard 152) (No in Step S4 of FIG. 2). In this case, the normal inputmode is executed (Step S6 of FIG. 2).

In the normal input mode, the user inputs characters “HATSUMEI”, “TARO”,“hatsumei”, “taro”, “2016”, “1”, and “1” to the input fields 210 212,220, 222, 230, 232, and 234 of the input screen 200 displayed on thedisplay unit 130, respectively, by using the input unit 150 (keyboard152). The user selects (designates) the input field 240 of the inputfields 240 and 242 of the input screen 200 displayed on the display unit130 using the input unit 150 (keyboard 152). As illustrated in FIG. 24,the screen processing unit 140 displays the input screen 200, in whichcharacters “HATSUMEI” and “TARO”, characters “hatsumei” and “taro” by afull width Katakana character, and characters “2016”, “1”, “1” by a halfwidth numeral, and the check mark are displayed at the input fields 210212, 220, 222, 230, 232, 234, and 240, respectively, on the display unit130.

Next, a handwriting input mode will be described. The screen processingunit 140 displays document data on the display unit 130 as the inputscreen 200 (see FIG. 20) (Step S2 of FIG. 2). Document data is datarepresenting a document by a hypertext markup language (HTML), aportable document format (PDF), or the like.

The user selects the handwriting input mode using the input unit 150(pointing device 154 or touch pen 156) (Yes in Step S4 of FIG. 2). Inthis case, the handwriting input mode is executed (Step S8 of FIG. 2).

In the handwriting input mode, the screen processing unit 140 capturesthe input screen 200 (see FIG. 20) displayed on the display unit 130 asa capture screen 300 (see FIG. 22), and saves the capture screen 300 inthe storage unit 120 and also displays the capture screen 300 on theinput screen 200 (Step S100 of FIG. 18).

As illustrated in FIG. 22, the user inputs characters “HATSUMEI”,“TARO”, “hatsumei”, “taro”, “2016”, “1”, and “1” to the input fields210, 212, 220, 222, 230, 232, and 234 of the capture screen 300 byhandwriting using the input unit 150 (pointing device 154 or touch pen156). The user inputs a character “v” and the like as a check mark tothe input field 240 of the input fields 240 and 242 of the capturescreen 300 by handwriting using the input unit 150 (pointing device 154or touch pen 156) (Step S102 of FIG. 18).

In this case, the user inputs characters “HATSUMEI”, “TARO”, “hatsumei”,“taro”, “2016”, “1”, “1”, and “v” by handwriting by extending beyond theinput fields 210, 212, 220, 222, 230, 232, 234, and 240 of the capturescreen 300. The stroke data processing unit 160 generates coordinatecolumns configuring characters, which are inputted to the capture screen300 by handwriting, as stroke data, respectively, and saves the strokedata in the storage unit 120.

The user performs a recognition instruction using the input unit 150(pointing device 154 or touch pen 156) (Step S104 of FIG. 18).

In this case, as illustrated in FIG. 22, the stroke data processing unit160 groups stroke data saved in the storage unit 120 into stroke datarepresenting a character string correlated with the input fields 210 and212, 220, 222, 230, 232, 234, and 240 and generates grouped stroke dataas pieces of grouping stroke data 310, 312, 320, 322, 330, 332, 334, and340 (Step S106 of FIG. 18).

The character recognition unit 170 conducts character recognition onpieces of grouping stroke data 310, 312, 320, 322, 330, 332, 334, and340 and converts each of pieces of grouping stroke data 310, 312, 320,322, 330, 332, 334, and 340 into each of the recognized characters 410,412, 420, 422, 430, 432, 434, and 440 as characters capable of beingedited by a computer (Step S108 of FIG. 18).

In this case, Katakana restriction information 520 is in correlationwith the input fields 220 and 222 of the coordinate table 500 and thus,the character recognition unit 170 recognizes that the input fields 220and 222 for causing the user to input a full width Katakana characterare present in the input screen 200 to convert the recognized characters420 and 422 into recognized characters 420 and 422 that represent a fullwidth Katakana character. Numeric restriction information 530 is incorrelation with the input fields 230, 232, and 234 of the coordinatetable 500 and thus, the character recognition unit 170 recognizes thatthe input fields 230, 232, and 234 for causing the user to input a halfwidth numeral are present in the input screen 200 to convert therecognized characters 430, 432, and 434 into the recognized characters430, 432, and 434 that represent a half width numeral. Check markattribute information 540 is in correlation with the input fields 240and 242 of the coordinate table 500 and thus, the character recognitionunit 170 recognizes that the input fields 240 and 242 for causing theuser to designate (select) the check mark are present in the inputscreen 200 to convert the recognized characters 440 into the recognizedcharacters 440 that represent the check mark.

In this case, as illustrated in FIG. 23, the screen processing unit 140displays the recognized characters 410, 412, 420, 422, 430, 432, 434,and 440 on the position at which input was performed to the capturescreen 300 by handwriting (Step S110 of FIG. 18). The recognizedcharacters 410, 412, 420, 422, 430, 432, 434, and 440 representcharacters “HATSUMEI” and “TARO”, characters “hatsumei” and “taro” by afull width Katakana character, characters “2016”, “1”, and “1” by a halfwidth numeral, and the character “v”, respectively.

For example, in a case where it is determined that there is no error inat least one recognized character of the recognized characters 410, 412,420, 422, 430, 432, 434, and 440 displayed on the capture screen 300 (Noin Step S112 of FIG. 18), the user executes Step S116 which will bedescribed later.

On the other hand, in a case where it is determined that there is anerror in at least one recognized character of the recognized characters410, 412, 420, 422, 430, 432, 434, and 440 displayed on the capturescreen 300 (Yes in Step S112 of FIG. 18), the user performs correctionon the at least one recognized character (Step S114 of FIG. 18).Specifically, Steps S102, S104, S106, S108, and S110 are executed forthe at least one recognized character.

The user performs an input instruction using the input unit 150(pointing device 154 or touch pen 156) (Step S116 of FIG. 18).

In a case where the coordinate table 500 is referenced and check markattribute information 540 is not correlated with the input fields 240and 242, the screen processing unit 140 recognizes that the input fields240 and 242 for attaching the check mark are not present in the inputscreen 200 (No in Step S120 of FIG. 19). In this case, similar to thesecond embodiment, Step S118 is executed. That is, the screen processingunit 140 displays the input screen 200 in which the recognizedcharacters 410, 412, 420, 422, 430, 432, and 434 are displayed at theinput fields 210, 212, 220, 222, 230, 232, and 234, respectively,instead of the capture screen 300 (Step S118 of FIG. 19). Characters“HATSUMEI” and “TARO”, characters “hatsumei” and “taro” by a full widthKatakana character, characters “2016”, “1”, and “1” by a half widthnumeral are displayed on the input fields 210, 212, 220, 222, 230, 232,and 234 of the input screen 200, respectively.

On the other hand, in a case where the coordinate table 500 isreferenced and check mark attribute information 540 is correlated withthe input fields 240 and 242, the screen processing unit 140 recognizesthat the input fields 240 and 242 for attaching the check mark arepresent in the input screen 200 (Yes in Step S120 of FIG. 19). In thiscase, as illustrated in FIG. 24, the screen processing unit 140 displaysthe input screen 200 in which the recognized characters 410, 412, 420,422, 430, 432, and 434 and the check mark are displayed at the inputfields 210, 212, 220, 222, 230, 232, 234, and 240, respectively, insteadof the capture screen 300 (Step S122 of FIG. 19). That is, characters“HATSUMEI” and “TARO”, characters “hatsumei” and “taro” by a full widthKatakana character, characters “2016”, “1”, and “1” by a half widthnumeral are displayed on the input fields 210, 212, 220, 222, 230, 232,and 234 of the input screen 200, respectively, and also the check markby which the input field 240 of the input fields 240 and 242 of theinput screen 200 is selected (designated) is displayed in the inputscreen 200.

As described above, the input display device 100 according to the thirdembodiment includes the display unit 130 that displays the input screen200 including the input fields 210, 212, 220, 222, 230, 232, 234, 240,and 242, the screen processing unit 140 that captures the input screen200 displayed on the display unit 130 as the screen for handwritinginput (capture screen 300) and displays the capture screen 300 on theinput screen 200, the stroke data processing unit 160 that groups strokedata when input was performed on the capture screen 300 by handwritinginto stroke data which represents the character strings, which are incorrelation with the input fields 210, 212, 220, 222, 230, 232, 234,240, and 242, and generates the stroke data as pieces of grouping strokedata 310, 312, 320, 322, 330, 332, 334, and 340, and the characterrecognition unit 170 that conducts character recognition on the piecesof grouping stroke data 310, 312, 320, 322, 330, 332, 334, and 340 andconverts the pieces of grouping stroke data 310, 312, 320, 322, 330,332, 334, and 340 into the recognized characters 410, 412, 420, 422,430, 432, 434, and 440. In this case, the character recognition unit 170converts the pieces of grouping stroke data 310, 312, 320, 322, 330,332, 334, and 340 into the recognized characters 410, 412, 420, 422,430, 432, 434, and 440 based on the input fields 210, 212, 220, 222,230, 232, 234, 240, and 242 of the input screen 200 and the screenprocessing unit 140 displays the recognized characters 410, 412, 420,422, 430, 432, 434, and 440 at the input fields 210, 212, 220, 222, 230,232, 234, and 240 of the input screen 200, respectively. For thatreason, in the input display device 100 according to the thirdembodiment, the user may collectively perform handwriting input withoutcausing the user to input the character string by handwriting every timeas in the technique described in Japanese Patent No. 3855696.

In the input display device 100 according to the third embodiment,attribute information (check mark attribute information 540) to beselected or designated by the user is correlated with at least two inputfields 240 and 242 among the input fields 210, 212, 220, 222, 230, 232,234, 240, and 242. The character recognition unit 170 converts pieces ofgrouping stroke data 310, 312, 320, 322, 330, 332, 334, and 340 into therecognized characters 410, 412, 420, 422, 430, 432, 434, and 440 basedon the input fields 210, 212, 220, 222, 230, 232, 234, 240, and 242 andattribute information (check mark attribute information 540). For thatreason, in the input display device 100 according to the thirdembodiment, for example, selection by the user (designations by theuser) may be displayed at the input fields 240 and 242, as attributeinformation.

Here, in the input display device 100 according to the third embodiment,even in a case where pieces of grouping stroke data 310, 312, 320, 322,330, 332, 334, and 340 are displaced with respect to the input fields210, 212, 220, 222, 230, 232, 234, and 240 of the input screen 200, thescreen processing unit 140 displays the recognized characters 410, 412,420, 422, 430, 432, 434, and 440 obtained by conducting characterrecognition on the pieces of grouping stroke data 310, 312, 320, 322,330, 332, 334, and 340 at the input fields 210, 212, 220, 222, 230, 232,234, and 240 based on the coordinate regions of the pieces of groupingstroke data 310, 312, 320, 322, 330, 332, 334, and 340, the coordinateregions of the input fields 210, 212, 220, 222, 230, 232, 234, and 240,and attribute information (check mark attribute information 540). Forexample, a portion of the coordinate regions of pieces of groupingstroke data 340 among the pieces of grouping stroke data 310, 312, 320,322, 330, 332, 334, and 340 overlaps the coordinate regions of the inputfield 240 among the input fields 210, 212, 220, 222, 230, 232, 234, and240, but a portion of the coordinate regions of a piece of groupingstroke data 340 among the pieces of grouping stroke data 310, 312, 320,322, 330, 332, 334, and 340 is extended beyond the coordinate regions ofthe input field 240 among the input fields 210, 212, 220, 222, 230, 232,234, and 240. Also, in this case, the recognized characters 440 obtainedby conducting character recognition on the piece of grouping stroke data340 may be displayed at the input field 240 based on the coordinateregions and attribute information (check mark attribute information540).

Fourth Embodiment

In the fourth embodiment, characters (for example, signature of a user)which are input to a portion other than respective input fields in thecapture screen 300 by handwriting may be displayed on the input screen200.

The operations of the input display device 100 according to the fourthembodiment will be described using FIGS. 18 to 21 and FIGS. 25 to 27.FIGS. 25 to 27 are diagrams for explaining the operations of the inputdisplay device 100 according to the fourth embodiment.

In the handwriting input mode, the screen processing unit 140 capturesthe input screen 200 (see FIG. 20) displayed on the display unit 130 asa capture screen 300 (see FIG. 25) and saves the capture screen 300 inthe storage unit 120, and also displays the capture screen 300 on theinput screen 200 (Step S100 of FIG. 18).

As illustrated in FIG. 25, the user inputs characters “HATSUMEI”,“TARO”, “hatsumei”, “taro”, “2016”, “1”, and “1” to the input fields210, 212, 220, 222, 230, 232, and 234 of the capture screen 300 byhandwriting using the input unit 150 (pointing device 154 or touch pen156). The user inputs a character “v” and the like as a check mark tothe input field 240 of the input fields 240 and 242 of the capturescreen 300 by handwriting using the input unit 150 (pointing device 154or touch pen 156). The user inputs a character obtained by encircling acharacter “HATSU”, as a signature of the user, in a portion other thanthe input fields 210 and 212, 220, 222, 230, 232, 234, 240, and 242 ofthe capture screen 300 (Step S102 of FIG. 18).

In this case, the user inputs characters “HATSUMEI”, “TARO”, “hatsumei”,“taro”, “2016”, “1”, “1”, and “v” by handwriting by extending beyond theinput fields 210, 212, 220, 222, 230, 232, 234, and 240 of the capturescreen 300. The user also inputs the character obtained by encircling acharacter “HATSU” to a right corner part of the capture screen 300 byhandwriting. The stroke data processing unit 160 generates coordinatecolumns configuring characters, which are inputted to the capture screen300 by handwriting, as stroke data, respectively, and saves the strokedata in the storage unit 120.

The user performs a recognition instruction using the input unit 150(pointing device 154 or touch pen 156) (Step S104 of FIG. 18).

In this case, as illustrated in FIG. 25, the stroke data processing unit160 generates stroke data representing character strings, which arecorrelated with the input fields 210, 212, 220, 222, 230, 232, 234, and240 among stroke data saved in the storage unit 120, as pieces ofgrouping stroke data 310, 312, 320, 322, 330, 332, 334, and 340. Thestroke data processing unit 160 generates stroke data representingcharacter strings, which are not correlated with the input fields 210and 212, 220, 222, 230, 232, 234, and 240 among stroke data saved in thestorage unit 120, as a handwritten character 350 (Step S106 of FIG. 18).

The character recognition unit 170 conducts character recognition onpieces of grouping stroke data 310, 312, 320, 322, 330, 332, 334, and340 and converts each of pieces of grouping stroke data 310, 312, 320,322, 330, 332, 334, and 340 into each of the recognized characters 410,412, 420, 422, 430, 432, 434, and 440 as characters capable of beingedited by a computer (Step S108 of FIG. 18).

In this case, Katakana restriction information 520 is in correlationwith the input fields 220 and 222 of the coordinate table 500 and thus,the character recognition unit 170 recognizes that the input fields 220and 222 for causing the user to input a full width Katakana characterare present in the input screen 200 to convert the recognized characters420 and 422 into the recognized characters 420 and 422 that represent afull width Katakana character. Numeric restriction information 530 is incorrelation with the input fields 230, 232, and 234 of the coordinatetable 500 and thus, the character recognition unit 170 recognizes thatthe input fields 230, 232, and 234 for causing the user to input a halfwidth numeral are present in the input screen 200 to convert therecognized characters 430, 432, and 434 into the recognized characters430, 432, and 434 that represent a half width numeral. Check markattribute information 540 is in correlation with the input fields 240and 242 of the coordinate table 500 and thus, the character recognitionunit 170 recognizes that the input fields 240 and 242 for causing theuser to designate (select) the check mark are present in the inputscreen 200 to convert the recognized character 440 into the recognizedcharacter 440 that represents the check mark.

In this case, as illustrated in FIG. 26, the screen processing unit 140displays the recognized characters 410, 412, 420, 422, 430, 432, 434,and 440 and the handwritten character 350 on the position at which inputwas performed to the capture screen 300 by handwriting (Step S110 ofFIG. 18). The recognized characters 410, 412, 420, 422, 430, 432, 434,and 440 represent characters “HATSUMEI” and “TARO”, characters“hatsumei” and “taro” by a full width Katakana character, characters“2016”, “1”, and “1” by a half width numeral, and the character “v”,respectively.

For example, in a case where it is determined that there is no error inat least one recognized character of the recognized characters 410, 412,420, 422, 430, 432, 434, and 440 displayed on the capture screen 300 (Noin Step S112 of FIG. 18), the user executes Step S116 which will bedescribed later.

On the other hand, in a case where it is determined that there is anerror in at least one recognized character of the recognized characters410, 412, 420, 422, 430, 432, 434, and 440 displayed on the capturescreen 300 (Yes in Step S112 of FIG. 18), the user performs correctionon the at least one recognized character (Step S114 of FIG. 18).Specifically, Steps S102, S104, S106, S108, and S110 are executed forthe at least one recognized character.

The user performs an input instruction using the input unit 150(pointing device 154 or touch pen 156) (Step S116 of FIG. 18).

In a case where the coordinate table 500 is referenced and check markattribute information 540 is not correlated with the input fields 240and 242, the screen processing unit 140 recognizes that the input fields240 and 242 for attaching the check mark are not present in the inputscreen 200 (No in Step S120 of FIG. 19). In this case, Step S118 isexecuted. That is, the screen processing unit 140 displays the inputscreen 200 in which the recognized characters 410, 412, 420, 422, 430,432, and 434 are displayed at the input fields 210, 212, 220, 222, 230,232, and 234, respectively, and the handwritten character 350 isdisplayed at the portion other than the input fields 210, 212, 220, 222,230, 232, and 234, instead of the capture screen 300 (Step S118 of FIG.19). Characters “HATSUMEI” and “TARO”, characters “hatsumei” and “taro”by a full width Katakana character, and characters “2016”, “1”, and “1”by a half width numeral are displayed on the input fields 210, 212, 220,222, 230, 232, and 234 of the input screen 200, respectively. Theencircled character “HATSU” is displayed on a right corner part of theinput screen 200.

On the other hand, in a case where the coordinate table 500 isreferenced and check mark attribute information 540 is correlated withthe input fields 240 and 242, the screen processing unit 140 recognizesthat the input fields 240 and 242 for attaching the check mark arepresent in the input screen 200 (Yes in Step S120 of FIG. 19). In thiscase, as illustrated in FIG. 27, the screen processing unit 140 displaysthe input screen 200 in which the recognized characters 410, 412, 420,422, 430, 432, and 434 and the check mark are displayed at the inputfields 210, 212, 220, 222, 230, 232, and 234, respectively, and thehandwritten character 350 is displayed at the portion other than theinput fields 210, 212, 220, 222, 230, 232, and 234, instead of thecapture screen 300 (Step S122 of FIG. 19). That is, characters“HATSUMEI” and “TARO”, characters “hatsumei” and “taro” by a full widthKatakana character, characters “2016”, “1”, and “1” by a half widthnumeral are displayed on the input fields 210, 212, 220, 222, 230, 232,234, and 240 of the input screen 200, respectively, and also the checkmark by which the input field 240 of the input fields 240 and 242 of theinput screen 200 is selected (designated) is displayed on the inputscreen 200. The encircled character “HATSU” is displayed on the rightcorner part of the input screen 200.

As described above, the input display device 100 according to the fourthembodiment includes the display unit 130 that displays the input screen200 including the input fields 210, 212, 220, 222, 230, 232, 234, 240,and 242, the screen processing unit 140 that captures the input screen200 displayed on the display unit 130 as the screen for handwritinginput (capture screen 300) and displays the capture screen 300 on theinput screen 200, the stroke data processing unit 160 that groups strokedata when input was performed on the capture screen 300 by handwritinginto stroke data which represents the character string, which are incorrelation with the input fields 210, 212, 220, 222, 230, 232, 234,240, and 242, and generates the stroke data as pieces of grouping strokedata 310, 312, 320, 322, 330, 332, 334, and 340, and the characterrecognition unit 170 that conducts character recognition on the piecesof grouping stroke data 310, 312, 320, 322, 330, 332, 334, and 340 andconverts the pieces of grouping stroke data 310, 312, 320, 322, 330,332, 334, and 340 into the recognized characters 410, 412, 420, 422,430, 432, 434, and 440. The screen processing unit 140 displays therecognized characters 410, 412, 420, 422, 430, 432, 434, and 440 at theinput fields 210, 212, 220, 222, 230, 232, 234, and 240 of the inputscreen 200. For that reason, in the input display device 100 accordingto the fourth embodiment, the user may collectively perform handwritinginput without causing the user to input the character string byhandwriting every time as in the technique described in Japanese PatentNo. 3855696.

In the input display device 100 according to the fourth embodiment, thestroke data processing unit 160 generates stroke data representingcharacter strings that are not correlated with the input fields 210,212, 220, 222, 230, 232, and 234 as the handwritten character 350. Thescreen processing unit 140 displays the handwritten character 350 andthe recognized characters 410, 412, 420, 422, 430, 432, and 434displayed at the input fields 210, 212, 220, 222, 230, 232, and 234 onthe input screen 200. For that reason, in the input display device 100according to the fourth embodiment, the character (for example,signature of a user), which is input to a portion other than respectiveinput field in the capture screen 300 by handwriting, may be displayedon the input screen 200.

Fifth Embodiment

In the second embodiment, although the input screen 200 includes, forexample, an input field for receiving a full width Katakana character oran input field for receiving a half width numeral as restrictioninformation that restricts a type of a character, the restrictioninformation is not limited to the input fields. The input screen 200 mayinclude, for example, an input field for receiving a half width alphabetas restriction information that restricts a type of a character, insteadof the input field for receiving a full width Katakana character.

Sixth Embodiment

In the first to fifth embodiments, the input screen 200 has thecoordinates (X0, Y0), (Xmax, Y0), (X0, Ymax), and (Xmax, Ymax), but isnot limited thereto. For example, the input screen 200 (window) ismovable on the display unit 130, and in a case where the input screen200 does not have coordinates on the display unit 130, when the userperforms handwriting input on the capture screen 300, the screenprocessing unit 140 may correlate the coordinates on the display unit130 with the input screen 200 by regarding the coordinates on thecapture screen 300 as event data.

Seventh Embodiment

As a seventh embodiment, a case where inputted characters (characterstring) are alphanumeric characters will be described. FIG. 28 is adiagram in which alphanumeric characters are input to a capture screen400 and the alphanumeric characters are displayed on an input screen. Itis assumed that a functional configuration and basic processing of theseventh embodiment are identical to the above-described embodiments, andspecific processing may be made by suitably referencing theabove-described embodiments. Accordingly, description will be mademainly on portions different from those of the above-describedembodiments in the seventh embodiment.

FIG. 28 illustrates the capture screen 400 in the handwriting inputmode. The capture screen 400 is obtained by capturing an input screenand input fields denoted as “First name”, “Last name”, and “Date ofBirth” are present in the input screen. A field of “Title” is alsopresent as a selectable selection button.

Here, the user inputs characters “John”, “Smith”, and “8/29/1970” to theinput field of the capture screen 400 by handwriting using the inputunit 150. The user inputs a check mark to the selection button byhandwriting. Coordinate columns that constitute the characters(character string) input by handwriting are respectively generated aspieces of stroke data and pieces of generated stroke data are saved inthe storage unit 120.

In this case, the pieces of stroke data that represent a characterstring correlated with the input field may be generated as groupingstroke data. Character recognition is conducted on the pieces of strokedata (grouping stroke data) and each of the pieces of stroke data isconverted into each of the recognized characters as characters(character string) capable of being edited by a computer.

Here, the character recognition unit 170 converts each of the pieces ofstroke data into the at least one recognized character by correlatingthe piece of stroke data with the input field. Specifically, if theinput field is the “First name” field, the character recognition unit170 performs character recognition of an alphanumeric character.Regarding the selection button, the character recognition unit 170recognizes the check mark as the recognized character to be recognized.

Although the character recognition unit 170 converts stroke data intothe at least one recognized character correlated with the input field,stroke data may be recognized in a predetermined language. Thepredetermined language may be set as the “English language”, “Japaneselanguage”, “Arabic”, and the like before corresponding processing isexecuted and the language which is set in the OS may be used.

FIG. 29 is the capture screen 400 indicating a state in which charactersand a character string are recognized. When the recognition is suitable,an operation for displaying is executed by the user and the at least onerecognized character and character string are input to each input field.That is, the at least one recognized character is displayed at theplurality of input field of the input screen.

Eighth Embodiment

An eighth embodiment will be described. In the fourth embodiment,handwriting input and other input in the Japanese language wasdescribed. The eighth embodiment has a form of classifying whethercharacters are to be recognized according to attribute information of afield to be used for receiving or whether stroke data is to be stored asit is. It is assumed that a functional configuration and basicprocessing of the eighth embodiment are identical to the above-describedembodiments, and specific processing may be made by suitably referencingthe above-described embodiments. Accordingly, description will be mademainly on portions different from those of the above-describedembodiments in the eighth embodiment.

FIG. 30 is a flowchart obtained by replacing processing of FIG. 18 inthe third embodiment. The same processing flow as that of FIG. 18 isassigned the same reference numerals and description thereof will beomitted. That is, Step S802 is added in FIG. 30.

In Step S802, it is determined whether attribute information of aposition of the input screen to which current stroke data is input isthe signature field or not (Step S802). Here, although in a case whereattribute information is not the signature field (No in Step S802),processing is executed from Step S108 similar to other embodiments, in acase where attribute information is the signature field (Yes in StepS802), character recognition is not performed. That is, stroke data isdisplayed on the input screen as a signature which is input byhandwriting.

Attribute information which is the signature may be stored in, forexample, the coordinate table 500 by being correlated with thesignature. For example, information “signature attribute informationsignature field” is stored for a content “signature field”.

FIG. 31 is a capture screen 510 based on the input screen in the eighthembodiment. In the capture screen 510, input fields are present,grouping stroke data 600 is input to, for example, the input field of“First name”, and is recognized as the recognized characters 610, asillustrated in FIG. 32.

Here, handwritten characters 602 which are input to the signature fieldis generated, as handwritten characters 612 as they are, based on anattribute of the signature field and is displayed.

As such, according to the eighth embodiment, the characters (strokedata) which are input to a normal input field by handwriting aregenerated as grouping stroke data and are recognized as a characterstring. The signature input to the signature field is generated andrecognized as the handwritten characters as they are.

In the eighth embodiment, although the signature field is described asan example, description made for the signature field may be applied toother fields (for example, a graphic input field and a check field) anda plurality of fields may be provided.

Ninth Embodiment

A ninth embodiment will be described. In the eighth embodiment,determination as to whether character recognition is to be performed ornot was performed based on the field to be used for receiving. In theninth embodiment, description will be made on a case where an attributeof the field to be for receiving is determined by further usingadditional coordinate information. It is assumed that a functionalconfiguration and basic processing of the ninth embodiment are identicalto the above-described embodiments, and specific processing may be madeby suitably referencing the above-described embodiments. Accordingly,description will be made mainly on portions different from those of theabove-described embodiments in the ninth embodiment.

FIG. 33 is a flowchart obtained by replacing processing of FIG. 30 (FIG.18) in the eighth embodiment (third embodiment). The same processingflow as that of FIG. 30 (FIG. 18) is assigned the same referencenumerals and description thereof will be omitted. That is, Steps S902and S904 are added in FIG. 33 in comparison with the operational flow ofFIG. 18.

In the capture screen, when handwriting input is performed, attributeinformation is acquired based on coordinate information which is inputby handwriting (Step S902). Here, when acquired attribute information isa target to be subjected to character recognition, processing isexecuted from Step S108 (Yes in Step S904). That is, characterrecognition is executed.

In contrast, when acquired attribute information is not present in thetarget to be subjected to character recognition, character recognitionis not performed (No in Step S904). Correlations of pieces of coordinateinformation and attribute information may be stored in, for example, thecoordinate table 500.

For example, FIG. 34 is a capture screen (input screen) based on theinput screen in which an input frame is not displayed. In this case, aninput region 750 is a region for which character recognition is notperformed and an input region 760 is a region for which characterrecognition is performed. That is, matters, which indicate a region forwhich character recognition is to be performed or not, are set inattribute information of coordinate information including the inputregion.

Accordingly, character recognition is performed on grouping stroke data700. For example, character recognition is performed on grouping strokedata 700 is subjected to character recognition and is displayed as therecognized characters 710 of FIG. 35.

Handwritten characters 702 which are input to the region for whichcharacter recognition is not performed are input as the handwrittencharacters as they are. That is, grouping stroke data 700 is displayedas the handwritten characters 712 in FIG. 35.

As such, according to the ninth embodiment, even in a case where theinput frame is not displayed, attribute information is suitably acquiredand the input stroke data is input as characters (character string),handwritten characters, or graphic according to attribute information.

Contents described in the seventh embodiment to the ninth embodiment arealso sufficiently described in the contents described in the firstembodiment to the sixth embodiment. In the first embodiment to the sixthembodiment, as the embodiments, although description was made mainly oncharacters such as Chinese characters and Katakana characters, mattersthat the description is also applied to alphanumeric characters aredescribed for confirmation. For example, also in FIG. 15, a numeral isrecognized and thus, matters described in FIG. 15 may be a range whichis enough for a person skilled in the art to understand.

Although an example of the alphanumeric character was described in theseventh embodiment to the ninth embodiment, it is needless to say thatother characters (for example, Arabic characters, Cyrillic characters,Thai characters) and symbols are also similarly applied.

MODIFICATION EXAMPLE

As described above, the disclosure is not limited to the above-describedembodiments and may be various modifications may be made thereto. Thatis, embodiments capable of being obtained by combining suitably modifiedtechnical units may also fall within a technical range of thedisclosure.

The present disclosure contains subject matter related to that disclosedin Japanese Priority Patent Application JP 2016-89589 filed in the JapanPatent Office on Apr. 27, 2016 and Japanese Priority Patent ApplicationJP 2017-44279 filed in the Japan Patent Office on Mar. 8, 2017, theentire contents of which are hereby incorporated by reference.

It should be understood by those skilled in the art that variousmodifications, combinations, subcombinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

What is claimed is:
 1. An input display device comprising: a displaythat displays a input screen including an input field for handwritinginput; a stroke data generator that generates grouping stroke data bygrouping stroke data which is included in handwriting input to the inputscreen; a character recognition processor that conducts characterrecognition on the grouping stroke data to convert the grouping strokedata into a recognized character; and an image processor that displaysthe recognized character on the display screen, wherein when there is ahandwriting input by a user in the input field, the characterrecognition processor conducts character recognition on the groupingstroke data generated from the handwriting input by the stroke datagenerator, and the image processor displays the recognized character onthe input field on the basis of the input field and attributeinformation of the input field.
 2. The input display device according toclaim 1, wherein the attribute information is at least information forselection or designation by a user of the input field.
 3. The inputdisplay device according to claim 1, wherein when the attributeinformation is information for selection of the input fieldcorresponding to the attribute information, the character recognitionprocessor converts the grouping stroke data into a recognized characterrepresenting selection.
 4. The input display device according to claim1, wherein the image processing unit displays the recognition characteron the position of the handwriting input.
 5. An input display method ofan input display device, the input method comprising: displaying a inputscreen including an input field for handwriting input; generatinggrouping stroke data by grouping stroke data which is included inhandwriting input to the input screen; conducting character recognitionon the grouping stroke data to convert the grouping stroke data into arecognized character; and displaying the recognized character on adisplay screen, wherein when there is a handwriting input by a user inthe input field, the character recognition is conducted on the groupingstroke data generated from the handwriting input by the stroke datagenerator, and the recognized character is displayed on the input fieldon the basis of the input field and attribute information of the inputfield.
 6. A non-transitory recording medium storing a program thatcauses a computer mounted on an input display device to execute aprocess comprising: displaying a input screen including an input fieldfor handwriting input; generating grouping stroke data by groupingstroke data which is included in handwriting input to the input screen;conducting character recognition on the grouping stroke data to convertthe grouping stroke data into a recognized character; and displaying therecognized character on a display screen, wherein when there is ahandwriting input by a user in the input field, the characterrecognition is conducted on the grouping stroke data generated from thehandwriting input by the stroke data generator, and the recognizedcharacter is displayed on the input field on the basis of the inputfield and attribute information of the input field.